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Abstract 

A simple iteration methodology for the solution of a set of a linear algebraic equations is pre- 
sented. The explanation of this method is based on a pure geometrical interpretation and pictorial 
representation. Convergence using this method is obtained and a simple numerical example is 
provided. 
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I. INTRODUCTION 



In various scientific fields there is a need to reduce the initial problem to a set of linear 
algebraic equations. This is true in physics when solving eigenstates or eigenfunction prob- 
lems in quantum mechanics. Also, it is applicable in applied physics and mathematics in 
the areas of linear programming, optimization methods and the root mean square method. 
Therefore this reduction of the initial problem to a set of linear algebraic equations can be 
viewed as a critical step in obtaining the correct solution to the initial problem. 

The methods used in obtaining a solution for a set of the linear algebraic equations are well 
known. Typically, the process begins with Cramer's formulas, deriving a final solution from 
the linear systems coefficients or similar numerical methods, then using a Gaussian method 
to obtain further improvement and modifications or even applying the Jacobi method to 
obtain further refinement. 

However, the real problem starts with the attempt to effectively solve a set of linear 
equations containing a large number of variables in which Cramer's method is not effective, 
because we need to calculate determinants, which in itself is a problem when a large number 
of variables are involved because there will be a loss of accuracy and the numerical stability 
of the calculation procedure is reduced. This would also apply in the application of both 
the Gaussian and Jacobi methods as well. In obtaining practical solutions we may also have 
observed linear algebraic set coefficients with error bars which makes the solution obtained 
even more sensitive to the method used in obtaining a numerical solution. 

In practice one of the main requirements desired in the numerical method selected is 
its stability and fast convergence in obtaining the final solution. In this paper the goal is 
to present a simple and rather convenient blueprint to describe the programming iteration 
method that could be used to solve the linear algebraic equation problem mentioned above. 



II. PROBLEM REFORMULATION. GEOMETRIC PICTURE 

In this section we want reformulate the algebraic problem involving the linear system 
solution using a geometrical language. Therefore, we need to obtain a number such as x^, 
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which satisfies the set of the linear system: 

Auxi + A 12 x 2 + ... + A ln x n = Bi 

A 2 lXl + A22X2 + ... + A 2n Xn = B 2 

A nl xi + A n2 x 2 + ... + A nn x n = B n (1) 
which can be written in a more convenient way, as: 

n 

A ikX k = Bi . (2) 

k=\ 

where A ik and Bi completely define the set of linear equations and have the following bounds 
1 < % < N. From a geometrical point of view we can consider every line in the set of linear 
equations, for any fixed number i, as a hyperplane in the n - dimensional space of the solution 
set. This means, that we can consider the solution as a vector, defined by X = (x±, X2, x n ), 

— * — * 

such that the matrix A ik transforms the vector X into the vector B which can be defined as 

— # 

B = (Bi,B 2 , ...,B n ). For instance, let's take some fixed point P E where en is a i-th 
hyperplane, such that P{x\,x\, ■■■,x^ l ) G Then for the arbitrary point Q G which 

also belongs to this hyperplane, we can write, that the scalar product Ai ■ QP = 0, where 
Ai, is defined as Ai = (An,Ai 2 , A in ) is _L to the (ai) vector. This case can be described 
as: 

n n n 

^2 A ik(x k -X P k ) = i > ^ A ik X k = ^ AikX k = Bi ■ ( 3 ) 

k=l k=l k=l 

Therefore, we can see, that from a geometrical point of view the solution of a linear set of 
algebraic equations is some point in n space, which is an intersection of all n hyperplanes. 
One of the possibly solutions therefore, is the construction of an iterative process, which will 
allow convergence at this point. 

III. NORMALIZATION PROCEDURE 

For the simplification of the numerical procedure, we can rewrite the set of the linear 
equations in the following way: 

n n 

J2 A lk x k = B t ^Y^ A 'ik x k = B[ . (4) 

k=l k=l 
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where we delineate, that: 

A ik = / sr n A2 = Aik ' B * = / sr n A2 = Bi ■ ^ 

V 2^,k=\ A ik V 2^,k=\ A ik 

Here we want to underline, that within the normalization procedure, the i index is fixed. 
Thus, we can now rewrite the set of the linear equations using the same form, but now our 
coefficients are normalized which means that for any indices pair of i, and k we observe: 
|Afe| < 1) which leads to the conclusion that all of our vectors which are perpendicular to 
their hyperplanes are interrelated. 



IV. A FIRST ARBITRARY POINT 



For the iteration procedure we need to choose the first point, from which the iteration 
procedure will start. It is obvious, that the iteration procedure convergence will be dependent 
on this starting point. It would be preferred to derive this point as close as possible to the 
final solution. Which of course is impossible to do in a general case. However, at a minimum 
we can require that within the iteration procedure itself that the chosen initial starting point 
must be numerically stable. In this case, we can start from a point which for example might 
belong to the first hyperplane and have coordinates: (x±, 0, 0, 0), where x\ = -J^-. 



V. AN ITERATION PROCEDURE 



For the start of the iteration procedure we need in that procedure a method for the 
projection of any arbitrary point A (defined by vector rr^) on the % -th hyperplane. In this 
case it is easy to see, that if the projection of this point is some point P (defined by vector 
x^), which belongs to the i-th hyperplane, then: 

x ( 2 ) = x {1) + XAi . (6) 

here A { is a unit vector, which is _L to the z-th hyperplane. Because P G (««), then we can 
state that: 

n n 

£ A ik {x {1) + \A ik ) = X + A ik4 ] = Bi (7) 

k=l k=l 

From where we determine: 

n 

\ = B i -J2 A ikX { k ) (8) 
k=i 



and it follows that 

n 

\Ai\ = l^ J2 A ikA ik = l (9) 
k=i 

After determination of the projection parameter A we can now describe the components of 
the projection point P: 

4 2) = 4 1} + ^ (io) 

where our iteration procedure consists of the following steps: 

1. The first step starts from the procedure of choosing the 1-st point, which will belong 
to the 1-st hyperplane and we will project it on to the next or second hyperplane. 

2. The second step consists of the projection of the 2-nd point which belongs to the 2-nd 
hyperplane, on to the next or 3-rd hyperplane and so forth until the last n hyperplane is 
reached. 

3. The third step then consists of the projection of the iteration point from the n-th 
hyperplane onto the first hyperplane, in other words the one which began our iteration 
procedure. After that we then can repeat all of the above listed procedures again until a 
complete convergence to the final solution is reached. 

VI. CONVERGENCE 

Proving the convergence in the numerical procedure described above can be accomplished 
in the following way. If we take some arbitrary initial point A, which belongs to a hyperplane 
and its projection point P on the i-th hyperplane, then it is clear, that the distance between 
projection point P and our final point of solution, noted as S, is: Lps = L^scosy? therefore 
for the k-th number of iterations we can to write, that: L k = L k _i costp k - Then the distance 
between solution S and our projection point P after n projections can be written as: 

Lp S = lim L (cos ipi cos tp 2 ... cos (11) 

For example, in the case, when all our hyperplanes are mutually orthogonal, then we have, 
for any number k a value of cos(/9 fc = and then we can automatically obtain a solution 
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after, at least for n — 1 iterations. However, in a case in which the difference between vectors, 
which happen to be _L to the hyperplanes is Aip^k — > for any two given hyperplanes then in 
this case the number of iteration procedures becomes n — > oo. It is clear, from a geometric 
point of view that in this case we have almost parallel hyperplanes and the convergence 
will proceed slowly. It is possible however, to roughly estimate the approximate number of 
iterations: 

L = L < cos ip e (12) 

from which we arrive at: 

2 ft 2 ln W / e \ 

< cos (p >= — / cos <pdip = > n m — . . = log 2 — (13) 



here e - is the accuracy of the solution. This example simply shows how the number of 
iterations depends on the choice of the first point selected for any arbitrary system, in 
general. However, the main result is that this iteration procedure is numerically stable, it is 
not dependent on the choice of the initial starting point to achieve convergence, if a solution 
to the set of linear systems exists. 

VII. NUMERICAL EXAMPLE 

Let's consider a simple numerical example. We will use our method to solve the following 
set of the linear equations: 

Xi + 2x 2 + 3x 3 + 4x 4 + 5x 5 = 55 
— 2xi + 10x 2 + x 3 + 3x 4 + 8x 5 = 73 
4xi + 5x 2 + 8x 3 + x 4 + 12x 5 = 102 

x\ + 2x2 + X3 + 3x4 + 10x 5 = 70 
8xi + 17x 2 + x 3 + 4x4 + 3x 5 = 76 (14) 

The exact solution for this set of the linear equations is: 

X\ = 1; x 2 = 2; x 3 = 3; x 4 = 4; x 5 = 5 (15) 

For the first starting point we have: 

X\ = 55; x 2 = 0; x 3 = 0; x 4 = 0; x 5 = (16) 
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TABLE I. Demonstration of the numerical procedure convergence. 



Number of iterations 





1 n 

1U oU 


ou 


( U 


1UU 


zuu 


1 


6.36729084 0.24225408 


1.00692936 


1.00648839 


0.99979049 


1.0 


X2 


-1.70446795 2.26483449 


2.011421866 


1.99653770 


2.00010026 


2.0 


X3 


-5.01380157 3.74870823 


3.02248973 


2.99148333 


3.00025466 


3.0 


X4 


13.2914602 4.28298482 


3.89357617 


4.00503095 


3.99990957 


4.0 


X5 


1.96186307 4.89304449 


5.03535119 


4.99844825 


5.00002627 


5.0 


A 


5.9 0.3677 


0.0377 


0.0046 


7.073 • 10~ 5 


1.178 • 10~ 9 



The result of this iteration procedure is presented in Table 1. Based on these results we are 
able to state that A = max\ Y^k=i A-ik%k — Bi\ is the maximum deviation from the solution 
at the given number of iterations when i G {1,2, 3, 4, 5}. 



VIII. CONCLUSION 

In the current mathematical literature there exists many different and useful methods for 
obtaining a solution for a set of the linear algebraic equations. The most notable include 
works by Cramer, Gauss, Jacobi, Gauss - Seidel. These methods provide a very useful 



solution to the problems described in the works [l-lSj]. However, a major problem arises in 
attempts to solve problems involving a very high number of variables, for example when 
the number of variables is equal to 10 4 — 10 6 or more. In this case we need to allow more 
time for convergence, especially if we use an iteration method. However, at the same time 
many of these existing methods become numerically unstable. For example, even a solution 
of obtaining the inverse matrix to becomes a non trivial problem. 

The application of our method is rather simple at the programming level and it has a 
transparent geometrical explanation and easy to understand pictorial representation. The 
efficiency of this method is much higher in solutions involving problems with large numbers 
of variables. The iteration procedure works rather quickly, for example in our case when the 
number of iterations is equal to 10 6 it takes less then 5 sec. In other methods the number of 
iterations required are sometimes much less than in our method, but they may require some 



special properties and/or conditions. This is the case of the Jacobi method. Our approach 
is not sensitive to these problems. 
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